Roll forward io expansion service to Java17#38974
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #38974 +/- ##
============================================
- Coverage 60.25% 57.72% -2.54%
+ Complexity 20572 13032 -7540
============================================
Files 3318 2513 -805
Lines 315452 261376 -54076
Branches 17142 10722 -6420
============================================
- Hits 190073 150868 -39205
+ Misses 116830 104796 -12034
+ Partials 8549 5712 -2837
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the Java version requirement for the expansion service from Java 11 to Java 17. Specifically, it updates the base Docker image to eclipse-temurin:17 and modifies the Gradle build configuration to require JavaVersion.VERSION_17, noting that Delta Lake requires Java 17+. There are no review comments, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
derrickaw
force-pushed
the
20260615_rollForwardIOExpansionJava17
branch
2 times, most recently
from
898e6d6 to
5ea0a53
Compare
derrickaw
force-pushed
the
20260615_rollForwardIOExpansionJava17
branch
from
5ea0a53 to
d8735a1
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request upgrades the Java environment requirements to Java 17 for the expansion service, updates the setup-java action to v4, and automatically configures Java home properties in the Gradle build configuration using environment variables. Feedback suggests refactoring the Gradle plugin logic to support ARM64 runners by checking both _X64 and _ARM64 environment variables, which can be simplified using a Groovy loop.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request facilitates the migration of the IO expansion service to Java 17. It includes necessary updates to build configurations, CI/CD environment setups, and container definitions to ensure compatibility with modern Java requirements and dependencies. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the Java environment configuration to support Java 17, including adding a Java 17 fallback installation in GitHub Actions, updating the expansion service Dockerfile base image to Java 17, and requiring Java 17 for the IO expansion service. Additionally, it configures Gradle to dynamically set JDK home properties and redirect JavaExec tasks to the specified JDK. Feedback points out that the Java 17 fallback installation path is overwritten and lost by the subsequent Java installation step, and suggests explicitly exporting the path. Another comment recommends adding a null check for testJavaHome before configuring JavaExec tasks to prevent build failures when the JDK path is unresolved.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Assigning reviewers: R: @ahmedabu98 for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
| if: ${{ inputs.java-version == 'default' }} | ||
| uses: actions/setup-java@v4 | ||
| with: | ||
| distribution: 'temurin' |
There was a problem hiding this comment.
Now java-version set to default means to install both Java11 and 17. This may work for fixing some test but makes future maintenance harder (more complex and implicit java version logic) and also making future breaking changes that effectively invalidate Java11 support (combine with the env fallback below) slips in
I would prefer make Java17 requirement explicit that is explicitly change java-version to "17 11" for the affected GHA workflows
There was a problem hiding this comment.
removed, thanks for the clarification
| def propName = "java${version}Home" | ||
| if (!project.hasProperty(propName)) { | ||
| def home = System.getenv("JAVA_HOME_${version}_X64") ?: | ||
| (System.getenv("JAVA_HOME_${version}_ARM64") ?: System.getenv("JAVA_HOME_${version}_arm64")) |
There was a problem hiding this comment.
Beam module plugin should be independent of GitHub Action specific arangement. Better avoiding further complicated fallback mechanism and/or making our build system tied to GitHub Action infra
Doesn't setting java17Home property in individual tests work?
There was a problem hiding this comment.
I forgot to verify in separate branch. All good now. Thanks.
https://github.com/apache/beam/actions/runs/27788956936/job/82232825777
derrickaw
force-pushed
the
20260615_rollForwardIOExpansionJava17
branch
from
c6bbc7d to
4381a8f
Compare
Abacn
left a comment
Abacn
left a comment
There was a problem hiding this comment.
Thanks!
Compared to the previous version, the change should be noop for most tests. We can fix foward if any other version tests need Java17+
Bumping the iceberg module to Java 17 (Iceberg 1.11.0 dropped Java 11) breaks the Java 11 modules that depend on it, because Gradle's JVM-version variant resolution refuses to let a Java 11 consumer depend on a Java 17 library. Raise requireJavaVersion 11 -> 17 in: - sdks/java/extensions/sql/iceberg - examples/java/iceberg The Java IO expansion service (sdks/java/io/expansion-service) also depends on IcebergIO but was already raised to Java 17 on master in apache#38974 (for Delta Lake), so it needs no change here. For apache#38925 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumping the iceberg module to Java 17 (Iceberg 1.11.0 dropped Java 11) breaks the Java 11 modules that depend on it, because Gradle's JVM-version variant resolution refuses to let a Java 11 consumer depend on a Java 17 library. Raise requireJavaVersion 11 -> 17 in: - sdks/java/extensions/sql/iceberg - examples/java/iceberg The Java IO expansion service (sdks/java/io/expansion-service) also depends on IcebergIO but was already raised to Java 17 on master in apache#38974 (for Delta Lake), so it needs no change here. For apache#38925 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cross-language wrapper validation launches the io expansion-service jar from Python via JAVA_HOME. That jar bundles IcebergIO, which is now Java 17 bytecode, so launching it on the default Java 11 fails with UnsupportedClassVersionError. createCrossLanguageUsingJavaExpansionTask was missing the testJavaHome -> JAVA_HOME redirect that apache#38974 added to the other cross-language task factories. Add it, and pass -PtestJavaVersion=17 in the precommit workflow so the expansion service runs on Java 17.
The cross-language wrapper validation launches the io expansion-service jar from Python via JAVA_HOME. That jar bundles IcebergIO, now Java 17 bytecode, so launching it on the default Java 11 fails with UnsupportedClassVersionError. createCrossLanguageUsingJavaExpansionTask was missing the JAVA_HOME redirect that apache#38974 added to the other cross-language task factories. Add it, defaulting to java17Home (already provided to this build) so bundled Java 17 IOs can be loaded.
The cross-language wrapper validation launches the io expansion-service jar from Python via JAVA_HOME. That jar bundles IcebergIO, now Java 17 bytecode, so launching it on the default Java 11 fails with UnsupportedClassVersionError. createCrossLanguageUsingJavaExpansionTask was missing the JAVA_HOME redirect that apache#38974 added to the other cross-language task factories. Add it, driven by -PtestJavaVersion (resolving java\Home), and pass -PtestJavaVersion=17 in the Xlang_Generated_Transforms precommit so the expansion service runs on a JDK that can load the bundled Java 17 IOs.
* Second try moving io expansion service to Java17 * fixing xlang issue * Fix cross-language CI by auto-discovering JDK homes and setting Java 17 as default * invert java 11 / 17 order installation * fix gemini * fix gemini * decouple BeamModulePlugin and setup-environment-action from GHA infra
* Second try moving io expansion service to Java17 * fixing xlang issue * Fix cross-language CI by auto-discovering JDK homes and setting Java 17 as default * invert java 11 / 17 order installation * fix gemini * fix gemini * decouple BeamModulePlugin and setup-environment-action from GHA infra
2 participants
First iteration that was reverted - #38931
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.